v33.0: Protocol Buffers v33.0

Compare Source

Announcements

• Protobuf News may include additional announcements or pre-announcements for upcoming changes.

Bazel

• Feat: update bazel central registry publish workflow (#​23465) (#​23913) (d5217fd)
• Add target_compatible_with parameter to proto_toolchain in Bazel rules (#​22429) (30d2332)
• Bazel: add missing rules_cc loads (#​23584) (d98e2ef)

Compiler

• Disable symbol visibility enforcement by default in C++ runtime (ae308fc)
• Ship all option dependencies to plugins along with regular ones. (abeb130)

C++

• Avoid calling deprecated arena-enabled constructors in arena.h. (813a7ef)
• Add a macro to make RepeatedField(Arena*) constructor private in a future release. (768db14)
• Add a macro to make Map(Arena*) constructor private in a future release. (543a17f)
• Optimize ReadPackedVarint (3d94d83)
• Add a macro to make RepeatedPtrField(Arena*) constructor private in a future release (6422b9d)
• Add IsEmpty() function to reflection. (b64e490)
• Refactor RuntimeAssertInBounds to remove repeated logic and make Get/Mutable easier to read. (2f270c4)
• Disable symbol visibility enforcement by default in C++ runtime (ae308fc)
• Fix a bug in the main C++ JSON parser/serializer camelcasing of certain non-style-compliant names incorrectly, in a way that would prevent it from interoperating with any other implementation on those fields. (e25e267)
• Fail early for messages with more than 65k fields. (90824aa)
• Add option to C++ JSON Parser/Serializer to allow customers to affirmatively disable legacy bug-compatibilty behaviors. (6ea1640)
• Fix mishandling on JSON serialization of Timestamp with invalid negative and too-large nanos value. (a959f27)
• Preserve features in type resolver (c7030f4)
• Add a DCHECK that ArenaStringPtr::Set(char*, Arena*) is not called with (95b1763)

Java

Restored compatibility of runtime with gencode created with protoc <3.21

With this release, compatibility of the runtime with older gencode down to 3.0.0 is restored, compared to the previous support minimum of gencode created with 3.22+. Note that it is still strongly recommended to regenerate your gencode with a newer protoc and to avoid using gencode which was created with an old protoc.

Generated code from this range is covered by CVE-2022-3171 and is potentially vulnerable to a Denial of Service risk.

JavaProto 4.x previously dropped compatibility with the potentially vulnerable generated code, having the behavior of:

• The vulnerable generated code was source-incompatible with new runtime (would not compile when built from source)
• The vulnerable generated code was ABI-incompatible with new runtime (when using a .class file compiled against old runtime, a NoSuchMethodException would be thrown at parse time).

Starting with this release:

• The vulnerable generated code is now source-compatible (will compile).
• The first time each potentially vulnerable type is parsed, an error message will be logged noting that potentially vulnerable generated code is in use and the name of the corresponding type.
• Environment variables may be set to either throw an exception instead (-Dcom.google.protobuf.error_on_unsafe_pre22_gencode) or to entirely silence the logged messages (-Dcom.google.protobuf.use_unsafe_pre22_gencode)

This change was made based on community feedback regarding the difficulty in identifying and quickly remediating stale gencode in their transitive dependencies weighed against a careful evaluation of the realistic risk exposure of DoS (with no risk of other concerns including information leak or RCE).

We strongly recommend that any users who observe the log messages to regenerate the corresponding code with a newer protoc. We recommend that any security-conscious services opt into the error_on_unsafe_pre22_gencode behavior to preclude any risk of a Denial of Service surface area being exposed.

A future 4.x release may flip the default behavior to error by default as a measure to further help the ecosystem avoid the Denial of Service risks, while still maintaining the ability to opt into continuing to use insecure gencode for users who are parsing trusted inputs and where the difficulty of regenerating is high.

Changes

• Switch the pre22 warning to use CopyOnWriteArraySet. (#​23969) (e55224c)
• Expose helpers for checking if messages and enums are nested. (8de4002)
• Fix a bug calculating the file name in the absense of directories. (c4ff7a6)
• Clarify the public APIs of GeneratorNames helpers. (537ac35)
• Expose helpers to predict generated class names in java. (eba6df2)
• Deprecate ClassName methods in favor of new QualifiedClassName ones. (ca4fb2f)
• Restore the 3-argument internalBuildGeneratedFileFrom. (4376591)
• Fix large java enums not being honored on lite runtime. (a995803)
• Slightly relax Java Poison Pill on prerelease versions (-rc1, -dev, etc). (7b0bee3)
• Avoid boxing/unboxing varint, fixed32, and fixed64 fields in UnknownFieldSet.Field (810272f)
• Readd new*List() methods on GeneratedMessageV3. (badaf41)
• Add Values.of(Map<String, Value> values). (c518f25)
• Fix handling of optional dependencies in java generator. (8d51e34)
• Restore ABI compatibility for extension methods which was previously (knowingly) broken with 4.x: 94a2a44 (ea33ae8)
• Restore Protobuf Java extension modifiers in gencode that were previously removed in 7bff169 (f2257f5)
• Ship all option dependencies to plugins along with regular ones. (abeb130)
• Optimize redaction state calculation (e05db5c)
• Add isPlaceholder() accessors to file, message, and enum descriptors (f978ec2)
• Improve Java gencode static initialization to avoid unnecessary temporaries again (745e15b)
• Improve Java gencode static initialization to avoid unnecessary temporaries (b68b673)
• Remove protobuf-util usages of guava except annotations. (5768acd)
• Restore compatibility of runtime with pre-3.22.x gencode impacted by CVE-2022-3171 (7c51e5b)
• Expose an iterator for GeneratedMessage.ExtendableMessage.extensions (b25d39e)

Rust

• Change Rust prelude to bring in traits as _ (c3f7e8d)
• Make message Muts Send (8bff944)
• See also UPB changes below, which may affect Rust.

Python

• Publish s390x wheels for Python/upb. (56b2b89)
• Fix a crash that happens during shutdown due to looking up modules in the cache (d57d270)
• Add construction support for repeated Timestamp/Duration/Struct/ListValue. (5f6c013)
• Fix handling of repeated extension fields in PyProto JSON (07ef676)
• Fixed a parser bug where closed enums are parsed incorrectly for non-repeated extensions. (c36f728)
• Fixed mypy errors by setting __slots__ to empty in .pyi files. (38ca2d3)
• Raise warnings for float_precision from python json_format. (4659cd7)
• Raise warnings when assign bool to int/enum field in Python Proto. This will turn into error in 34.0 release. (4ee55d7)

PHP

• Fix(php): php errors on repeated field (#​23372) (6fee29b)

UPB (Python/PHP/Ruby C-Extension)

• Fixed a parser bug where closed enums are parsed incorrectly for non-repeated extensions. (c36f728)

Other

• Update token for BCR release to reuse existing BOT_ACCESS_TOKEN used for staleness_refresh.yml and update_php_repo.yml (#​23925) (dcace2f)
• Use the 'better' JSON parser on the conformance suite harness. (4b4e405)
• Add JSON conformance test that a single value provided for a repeated field should parse fail. (9806994)
• Add conformance test cases for malformed nanos fields on Durations and Timestamps. (a6bdd0a)